Image processing device, setting support method, and non-transitory computer-readable media

ABSTRACT

An image processing device is provided. An image processing device includes: a program executing part that applies parameter groups to an image processing program which searches a search object using an image group with different resolutions created from an input image, acquires a search result for each of the parameter groups, and acquires an image group which is acquired in the course of execution of the image processing program for each parameter group; a synthesis part that creates a synthetic image from each of a plurality of acquired image groups; a display part that displays the plurality of created synthetic images and displays the plurality of acquired search results to overlap the plurality of synthetic images; and a setting part that sets the parameter group corresponding to the synthetic image selected from the plurality of synthetic images displayed on the display part by an operation in the image processing program.

CROSS-REFERENCE TO RELATED APPLICATION

This application claims the benefit of Japan patent No. 2017-165230,filed on Aug. 30, 2017. The entirety of the above-mentioned patentapplication is hereby incorporated by reference herein and made a partof this specification.

BACKGROUND Technical Field

The disclosure relates to a technique of supporting setting ofparameters for an image processing program for searching an object froman image.

Description of Related Art

Recently, various image processing programs for searching an object froman image have been developed. For example, Japanese Laid-open No.2017-041190 (Patent Document 1) discloses a method of detecting aworkpiece from an image using an image processing program employing acoarse-to-fine search method. The coarse-to-fine search method is analgorithm of creating an image group having different resolutions froman input image, sequentially searching a search object from an imagewith a lowest resolution, and searching an object while restricting asearch range in an image with a next higher resolution based on thesearch results for the images.

A user needs to set various parameters for an image processing programemploying a coarse-to-fine search method to adjust search accuracy, aprocessing time, or the like. At this time, the user needs to understanddetails of the image processing program. However, since internalprocesses are covered by a block box, the user cannot easily understandwhat parameters need to be set and in what way. When the number of typesof parameters to be set increases, it is more difficult to adjust theparameters. Accordingly, there is demand for a technique of supportingsetting of parameters for an image processing program for searching anobject from an image.

SUMMARY

According to an embodiment, an image processing device includes: animage acquiring part that acquires an input image including a searchobject; a program executing part that applies one or more predeterminedparameter groups to an image processing program which searches thesearch object using an image group with different resolutions which iscreated from the input image, acquires a search result from the imageprocessing program by applying each of the one or more parameter groups,and acquires the image group which is acquired in the course ofexecution of the image processing program for each of the one or moreparameter groups; a synthesis part that creates a synthetic image fromeach of the acquired one or more image groups; a display part thatdisplays the created one or more synthetic images and displays theacquired one or more search results to overlap the one or more syntheticimages; an operation receiving part that receives an operation ofselecting any one of the one or more synthetic images displayed on thedisplay part; and a setting part that sets the parameter groupcorresponding to the synthetic image selected by the operation in theimage processing program.

According to an embodiment, a setting support method of supportingsetting of parameters for an image processing program includes: a stepof acquiring an input image including a search object; a step ofapplying one or more predetermined parameter groups to an imageprocessing program which searches the search object using an image groupwith different resolutions which is created from the input image,acquiring a search result from the image processing program for each ofthe one or more parameter groups, and acquiring the image group which isacquired in the course of execution of the image processing program foreach of the one or more parameter groups; a step of creating a syntheticimage from each of the acquired one or more image groups; a step ofdisplaying the created one or more synthetic images and displaying theacquired one or more search results to overlap the one or more syntheticimages on a display part; a step of receiving an operation of selectingone of the one or more synthetic images displayed on the display part;and a step of setting the parameter group corresponding to the syntheticimage selected by the operation in the image processing program.

According to an embodiment, a setting support program for supportingsetting of parameters for an image processing program causes an imageprocessing device to perform: a step of acquiring an input imageincluding a search object; a step of applying one or more predeterminedparameter groups to an image processing program which searches thesearch object using an image group with different resolutions which iscreated from the input image, acquiring a search result from the imageprocessing program for each of the one or more parameter groups, andacquiring the image group which is acquired in the course of executionof the image processing program for each of the one or more parametergroups; a step of creating a synthetic image from each of the acquiredone or more image groups; a step of displaying the created one or moresynthetic images and displaying the acquired one or more search resultsto overlap the one or more synthetic images on a display part; a step ofreceiving an operation of selecting one of the one or more syntheticimages displayed on the display part; and a step of setting theparameter group corresponding to the synthetic image selected by theoperation in the image processing program.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a diagram illustrating an example of an entire configurationof an image processing system according to an embodiment.

FIG. 2 is a conceptual diagram schematically illustrating support ofsetting of parameters for a pyramid algorithm.

FIG. 3 is a diagram illustrating an example of a data structure of aparameter group.

FIG. 4 is a conceptual diagram schematically illustrating a pyramidalgorithm.

FIG. 5 is a conceptual diagram schematically illustrating a pyramidimage synthesizing process which is performed by a synthesis part.

FIG. 6 is a diagram illustrating an example of a display mode in adisplay part.

FIG. 7 is a conceptual diagram schematically illustrating a synthesizingprocess according to a first modified example.

FIG. 8 is a conceptual diagram schematically illustrating a synthesizingprocess according to a second modified example.

FIG. 9 is a conceptual diagram schematically illustrating a synthesizingprocess according to a third modified example.

FIG. 10 is a diagram schematically illustrating a synthesizing processof a pixel group.

FIG. 11 is a diagram schematically illustrating an example of a hardwareconfiguration of an image processing device according to the embodiment.

FIG. 12 is a flowchart schematically illustrating a part of processeswhich are performed by the image processing device according to theembodiment.

DESCRIPTION OF THE EMBODIMENTS

According to the embodiments, it is possible to support setting ofparameters for an image processing program for searching an object froman image.

Hereinafter, embodiments of the invention will be described withreference to the accompanying drawings. In the following description,the same components and elements will be referenced by the samereference signs and have the same names and functions. Accordingly,detailed description thereof will not be repeated. Embodiments andmodified examples which will be described below may be appropriatelyselectively combined.

A. Configuration of Image Processing System 1

An entire configuration of an image processing system 1 will bedescribed below with reference to FIG. 1. FIG. 1 is a diagramillustrating an example of the entire configuration of the imageprocessing system 1 according to an embodiment.

The image processing system 1 includes an image processing device 100that is also referred to as a visual sensor, a programmable logiccontroller (PLC) 5 that can communicate with the image processing device100, and an imaging part 8 (an image acquiring part) that is connectedto the image processing device 100.

The PLC 5 controls the transport mechanism 6 and the like in cooperationwith the image processing device 100. The imaging part 8 of an examplefurther includes a shooting device which are partitioned into aplurality of pixels such as a charge coupled device (CCD) or acomplementary metal oxide semiconductor (CMOS) sensor in addition to anoptical system such as a lens. The imaging part 8 sequentially imagesworkpieces W which are transported by the transport mechanism 6. Aninput image which is acquired by shooting by the imaging part 8 istransmitted to the image processing device 100.

The image processing device 100 searches a workpiece W in the inputimage acquired from the imaging part 8 by executing a predeterminedimage processing program for searching an object from an image. Forexample, the image processing device 100 searches a workpiece W from theinput image by an image processing program using a coarse-to-fine searchmethod. Hereinafter, the image processing program using a coarse-to-finesearch method is also referred to as a “pyramid algorithm.” Althoughdetails will be described later, the pyramid algorithm creates an imagegroup with different resolutions from the input image and searches thesearch object sequentially from an image with the lowest resolution inthe created image group. At this time, the pyramid algorithm searches anobject while restricting a search range in the image with the nexthigher resolution on the basis of the search result for each image.Hereinafter, the image group with different resolutions which isacquired in the course of execution of the pyramid algorithm is alsoreferred to as a “pyramid image.”

The image processing device 100 executes an image processing program forinspection of a defect or stain, measurement of a size, an arrangementdirection, and the like of workpieces W, and recognition of charactersor figures on the surface of each workpiece W on a workpiece searchedfrom the input image. The execution result is displayed, for example, ona display part 102 of the image processing device 100.

B. Parameter Setting Support

A method of supporting setting of parameters for the pyramid algorithmwill be described below with reference to FIGS. 2 and 3.

FIG. 2 is a conceptual diagram schematically illustrating support ofsetting of parameters for the pyramid algorithm. As illustrated in FIG.2, the image processing device 100 includes a display part 102, acontrol device 110, and a storage device 120 as principal hardware.

The control device 110 controls the image processing device 100. Thecontrol device 110 includes a program executing part 151 that executesan image processing program 20 and a synthesis part 152 that synthesizesa pyramid image which is created in the course of execution of thepyramid algorithm.

The storage device 120 includes an image processing program 20 forexecuting a pyramid algorithm, setting parameters 22 that are currentlyset in the image processing program 20, parameter groups 28A to 28C thatserve as candidates which are applied to the image processing program20, an input image 30 which is acquired by imaging a workpiece W, and atemplate group 31 that is used to search the workpiece W from the inputimage 30.

In the following description, the parameter groups 28A to 28C arecollectively referred to as parameter groups 28. A “parameter group”mentioned herein refers to a set of parameters which are required forexecuting the image processing program 20. FIG. 3 is a diagramillustrating an example of a data structure of a parameter group 28. Asillustrated in FIG. 3, the parameter group 28 includes a searchthreshold value group that is used to determine whether a search objectis included in each image as a threshold value group which is applied toeach image of the pyramid image. The parameter group 28 also includes anedge threshold value group that is used to determine whether each pixelof each image is a pixel indicating an edge as a threshold value groupwhich is applied to each image of the pyramid image. In addition,various parameters which are used at the time of execution of thepyramid algorithm can be defined in the parameter group 28. A method ofusing the parameter group 28 will be described later.

Referring back to FIG. 2, the template group 31 includes templates 31Ato 31C. The templates 31A to 31C may be reference images which areacquired by imaging a workpiece W in advance, or may be features whichare extracted as feature portions of the workpiece W from the referenceimages.

The image processing device 100 according to this embodiment supportssetting of parameters for the pyramid algorithm. More specifically, theprogram executing part 151 applies each of the predetermined parametergroups 28A to 28C to the image processing program 20 for executing thepyramid algorithm, acquires a search result using the image processingprogram 20 for each parameter group, and acquires pyramid images 32A to32C which are acquired in the course of execution of the imageprocessing program 20 for each parameter group.

Thereafter, the synthesis part 152 creates synthetic images 34A to 34Cfrom the pyramid images 32A to 32C. The display part 102 displays thecreated synthetic images 34A to 34C and displays search results 33A to33C for each parameter group to overlap the synthetic images 34A to 34C.

For example, it is assumed that a workpiece W has not been searched fromthe input image 30 using the parameter group 28A and a workpiece W hasbeen searched from the input image 30 using the parameter groups 28B and28C. In this case, the image processing device 100 displays the searchresults 33B and 33C to overlap only the synthetic images 34B and 34Ccorresponding to the parameter groups 28B and 28C. Accordingly, a usercan easily determine whether each of the parameter groups 28A to 28C isbetter.

The image processing device 100 is configured to select any one of thedisplayed synthetic images 34A to 34C. This selecting operation isreceived, for example, by an operation receiving part (for example, anarbitrary input device such as a mouse, a keyboard, and a touch panel)of the image processing device 100. The image processing device 100 setsthe parameter group corresponding to the synthetic image selected by theuser's operation as the setting parameter 22 for the image processingprogram 20. In the example illustrated in FIG. 2, the synthetic image34B is selected and the parameter group 28B corresponding to thesynthetic image 34B is set as the setting parameter 22.

As described above, a user can intuitively understand whether eachparameter group is better by visually checking the synthetic imagescreated for the parameter groups and the search results for theparameter groups. Accordingly, the user can select one parameter groupwith which good search accuracy can be achieved. As a result, a user nothaving any knowledge of the image processing program 20 can performsetting of an optimal parameter group and can support setting ofparameters for the image processing program 20.

In one or some exemplary embodiments, the display part 102 additionallydisplays evaluated values for the parameter groups 28A to 28C. As theevaluated values, for example, execution times when the parameter groups28A to 28C are applied to the image processing program 20 are displayed.More specifically, the image processing device 100 has a clockingfunction of a counter or the like and stores an execution time fromstart of the image processing program to end of the image processingprogram for each parameter group. The display part 102 displays theexecution time for each parameter group in parallel to the correspondingsynthetic image.

In general, the execution time is extended as search accuracy increases,and search accuracy decrease as the execution time is shortened. A usercan visually compare such contradictory indices and can select oneparameter group in comprehensive consideration of the search accuracyand the execution time. Accordingly, the user can easily set an optimalparameter group that can realize desired search accuracy and a desiredexecution time.

An example in which the execution time of the image processing program20 is displayed as the evaluated value for each parameter group has beendescribed above, the evaluated value to be displayed is not limited tothe execution time. For example, search accuracy or a detection rate ofa search object or the like when each parameter group is applied to theimage processing program 20 may be displayed as the evaluated value foreach parameter group. More specifically, the image processing device 100creates a plurality of artificial images on the basis of the input image30. An artificial image is created, for example, by changing pixelpositions of the input image 30 or applying a predeterminedenvironmental variation to the input image 30. The image processingdevice 100 performs a search process on the created artificial imagesfor the parameter groups. The image processing device 100 identifies thesearch accuracy or the detection rate by collecting the search resultsin the artificial images for each parameter group.

FIG. 2 illustrates an example in which three parameter groups 28 arestored, but the number of parameter groups which are prepared in advanceis not particularly limited. The number of parameter groups which areprepared in advance is equal to or greater than two.

C. Pyramid Algorithm

The pyramid algorithm which is executed by the program executing part151 will be described below with reference to FIGS. 2 and 4. FIG. 4 is aconceptual diagram schematically illustrating a pyramid algorithm.

As illustrated in FIG. 4, the program executing part 151 creates animage group with different resolutions (that is, a pyramid image 32A)from the input image 30. In other words, the program executing part 151reduces the input image 30 by a predetermined magnification (forexample, a half times) and creates an image at each reduction ratio asthe pyramid image 32A. In the example illustrated in FIG. 4, the pyramidimage 32A is created from the input image 30, and the pyramid image 32Aincludes images 41 to 43 (an image group) with different resolutions.

The program executing part 151 searches a search object sequentiallyfrom the image with a lowest resolution among the images 41 to 43. Morespecifically, the program executing part 151 acquires a template 31Acorresponding to the resolution of the image 41 from the template group31. The program executing part 151 scans the image 41 with the template31A and calculates a degree of similarity between each area in the image41 and the template 31A. An arbitrary algorithm such as sum of squaredifference (SSD), sum of absolute difference (SAD), normalizedcross-correlation (NCC), or zero-mean normalized cross-correlation(ZNCC) is employed as the method of calculating the degree ofsimilarity. The program executing part 151 identifies an image area ofwhich the degree of similarity is greater than the search thresholdvalue th1 of the parameter group 28A among the image areas in the searchrange, and sets the image area as a candidate including the searchobject. In the example illustrated in FIG. 4, the image areas 41A and41B are identified as candidates of the image area including the searchobject.

Then, the program executing part 151 acquires a template 31Bcorresponding to the resolution of the image 42 from the template group31. Thereafter, the program executing part 151 sets image areas 42A and42B corresponding to the image areas 41A and 41B which are specified ascandidates including the search object as a search range. The programexecuting part 151 program executing part 151 scans the image areas 42Aand 42B with the template 31B and calculates degrees of similaritybetween areas in the image areas 42A and 42B and the template 31B. Theprogram executing part 151 identifies an image area in which the degreeof similarity is greater than a search threshold value th2 of theparameter group 28A among the image areas in the search range and setsthe image area as a candidate including the search object. In theexample illustrated in FIG. 4, the image area 42A is identified as acandidate including the search object.

Then, the program executing part 151 acquires a template 31Ccorresponding to the resolution of the image 43 from the template group31. Thereafter, the program executing part 151 sets an image area 43Acorresponding to the image area 42A which is specified as a candidateincluding the search object as a search range. The program executingpart 151 scans the image area 43A with the template 31C and calculatesdegrees of similarity between areas in the image area 43A and thetemplate 31C. The program executing part 151 identifies an image area inwhich the degree of similarity is greater than a search threshold valueth3 of the parameter group 28A among the image areas in the search rangeand sets the image area as a candidate including the search object.

In this way, the program executing part 151 searches a search objectsequentially from the image with a lowest resolution among the images ofthe pyramid image, and searches the search object while restricting thesearch range in an image with a next higher resolution on the basis ofthe search results in the images. A finally left candidate is output asthe search result. For example, the search result is expressed ascoordinate values in the input image 30.

In the example illustrated in FIG. 2, the program executing part 151outputs the pyramid image 32A and the search result 33A as a result ofapplication of the parameter group 28A to the image processing program20. Similarly, the program executing part 151 outputs the pyramid image32B and the search result 33B as a result of application of theparameter group 28B to the image processing program 20. Similarly, theprogram executing part 151 outputs the pyramid image 32C and the searchresult 33C as a result of application of the parameter group 28C to theimage processing program 20. The pyramid images 32A to 32C are output tothe synthesis part 152 and the search results 33A to 33C are output tothe display part 102.

D. Synthesizing Process

A process of synthesizing a pyramid image which is performed by thesynthesis part 152 will be described below with reference to FIGS. 2, 5,and 6. FIG. 5 is a conceptual diagram schematically illustrating theprocess of synthesizing a pyramid image which is performed by thesynthesis part 152.

In the example illustrated in FIG. 5, a synthesizing process isperformed on a pyramid image in which a person appears, but thesynthesizing process may be performed on a pyramid image in which aworkpiece appears.

Images 41 to 44 which are created from the input image 40 areillustrated in FIG. 5. The images 41 to 44 are acquired by applying anedge extracting filter such as a differential filter to a pyramid imagecreated from the input image 40. In the example illustrated in FIG. 5,the images 41 to 44 are binarized into an edge portion and a non-edgeportion on the basis of edge threshold values thA to thD of theparameter groups 28, where the edge portion is expressed in white andthe non-edge portion is expressed in black. Typically, a pixel value ofthe edge portion is defined as “1” and a pixel value of the non-edgeportion is defined as “0.”

The synthesis part 152 creates an intermediate image 45 from the images41 to 43, and creates a synthetic image 46 from the intermediate image45 and the image 44. More specifically, the synthesis part 152 adjuststhe sizes of the images 41 to 43 and then synthesizes pixel values atthe same coordinates to create the intermediate image 45. Thesynthesizing process employs an arbitrary operation such as an ANDoperation, an OR operation, or an addition operation.

Thereafter, the synthesis part 152 adjusts the sizes of the image 44 andthe intermediate image 45 and synthesizes the image 44 and theintermediate image 45 to create a synthetic image 46. At this time, thesynthesis part 152 creates the synthetic image 46 such that differencesin pixel values between the image 44 and the intermediate image 45 aredisplayed. Edges are likely to be lost in the images 41 to 43 with lowresolutions. For the synthetic image 46, the synthetic image 46 can begenerated by identifying the lost edges. For example, the pixel value inthe synthetic image 46 is “2” in portions indicating a valid edge inboth the image 44 and the intermediate image 45, is “1” in portionsindicating a valid edge in the image 44 and indicating an invalid edgein the intermediate image 45, and is “0” in portions having an invalidedge in both the image 44 and the intermediate image 45.

E. Display Process

A display process which is performed by the display part 102 will bedescribed below with reference to FIG. 6. FIG. 6 is a diagramillustrating a display mode of the display part 102.

The display part 102 displays synthetic images 46 to 48 which arecreated for the parameter groups 28A to 28C by the synthesis part 152.As described above, the synthetic images 46 to 48 are created such thatedges lost in the course of execution of the pyramid algorithm can beidentified. The display part 102 displays the synthetic images 46 to 48in a mode in which such lost edge portions can be identified. In theexample illustrated in FIG. 6, an edge portion not lost in the course ofexecution of the pyramid algorithm is displayed in white, and an edgeportion lost in the course of execution of the pyramid algorithm isdisplayed in gray, and the other portions are displayed in black. Thisdifference may be displayed in different colors. By this display, a usercan easily understand a reason for a bad search result and can easilydetermine suitability of each parameter group.

The display part 102 displays the search result of the pyramid algorithmfor each parameter group on the corresponding synthetic image. Forexample, it is assumed that a search object (for example, an eye) hasnot been searched with the parameter group 28A and the search object hasbeen searched with the parameter groups 28B and 28C. In this case, thedisplay part 102 does not display the search result for the syntheticimage 46 corresponding to the parameter group 28A and displays searchresults 47A and 48A on only the synthetic images 47 and 48 correspondingto the parameter groups 28B and 28C.

F. Modified Example

FIG. 5 illustrates an example in which the synthesis part 152 createsthe intermediate image 45 from some images 41 to 43 of the pyramid imageand creates the synthetic image 46 from the other image 44 and theintermediate image 45, but the synthesis method which is performed bythe synthesis part 152 is not limited to the example illustrated in FIG.5. For example, the synthesis part 152 may create the synthetic image 46without creating the intermediate image 45. In this case, the synthesispart 152 adjusts the resolutions of the images constituting the pyramidimage and then creates a synthetic image from the image group.

Hereinafter, first and second modified examples of the synthesizingprocess which is performed by the synthesis part 152 will besequentially described.

FIG. 7 is a conceptual diagram schematically illustrating a synthesizingprocess according to the first modified example. In FIG. 7, images 51 to54 obtained by inversely converting the pyramid image (for example, theimages 41 to 44) with a predetermined same resolution (a same size) areillustrated. The images 51 to 54 are binarized into an edge portion anda non-edge portion, where the edge portion is displayed in white and thenon-edge portion is hatched. Typically, the pixel value of the edgeportion is defined as “1” and the pixel value of the non-edge portion isdefined as “0.”

The synthesis part 152 performs an AND operation on the pixels values atthe same coordinates in the binarized images 51 to 54 and creates asynthetic image 46-1 as the result of the AND operation. For example,paying attention to pixels 51A to 54A at the same coordinates in theimages 51 to 54, since the pixel value of the pixel 51A is “0,” thepixel value of the pixel 52A is “0,” the pixel value of the pixel 53A is“1,” and the pixel value of the pixel 54A is “1,” the result of the ANDoperation is “0.” As a result, a pixel 56 in the synthetic image 46-1corresponding to the pixels 51A to 54A has a pixel value of “0.” Thesynthesis part 152 performs this AND operation on all the pixels.

In this way, the synthesis part 152 performs the AND operation on thepixels at the same coordinates in the images 51 to 54 of the sameresolutions and outputs a synthetic image 46-1 as the result of the ANDoperation. A user can easily understand a portion indicating a validedge in all the layers by confirming the synthetic image 46-1 subjectedto the AND operation. Alternatively, a user can easily understand aportion indicating an invalid edge in some of the layers.

FIG. 8 is a conceptual diagram schematically illustrating a synthesizingprocess according to the second modified example. The same images 51 to54 as illustrated in FIG. 7 are illustrated in FIG. 8.

In this modified example, the synthesis part 152 performs an ORoperation on the pixels values at the same coordinates in the binarizedimages 51 to 54 and creates a synthetic image 46-2 as the result of theOR operation. For example, paying attention to pixels 51A to 54A at thesame coordinates in the images 51 to 54, since the pixel value of thepixel 51A is “0,” the pixel value of the pixel 52A is “0,” the pixelvalue of the pixel 53A is “1,” and the pixel value of the pixel 54A is“1,” the result of the OR operation is “1.” As a result, a pixel 56 inthe synthetic image 46-2 corresponding to the pixels 51A to 54A has apixel value of “1.” The synthesis part 152 performs this OR operation onall the pixels.

In this way, the synthesis part 152 performs the OR operation on thepixels at the same coordinates in the images 51 to 54 of which theresolutions are arranged and outputs a synthetic image 46-2 as theresult of the OR operation. A user can easily understand a portionindicating a valid edge in part by confirming the synthetic image 46-2subjected to the OR operation. Alternatively, a user can easilyunderstand a portion indicating an invalid edge in all the layers.

FIG. 9 is a conceptual diagram schematically illustrating a synthesizingprocess according to the third modified example. The same images 51 to54 as illustrated in FIG. 7 are illustrated in FIG. 9.

In this modified example, the synthesis part 152 synthesizes the pixelsvalues at the same coordinates in the images 51 to 54 of the sameresolutions such that information of the pixel values is not lost, andoutputs a synthetic image 46-3 as the synthesis result. Morespecifically, bit strings with at least the same number of bits as inthe images 51 to 54 are allocated to the pixels of synthetic image 46-3.A pixel value of a corresponding pixel in the image 51 corresponding toa first layer of the pyramid image is reflected in the first bit of eachpixel of the synthetic image 46-3. A pixel value of a correspondingpixel in the image 52 corresponding to a second layer of the pyramidimage is reflected in the second bit of each pixel of the syntheticimage 46-3. A pixel value of a corresponding pixel in the image 53corresponding to a third layer of the pyramid image is reflected in thethird bit of each pixel of the synthetic image 46-3. A pixel value of acorresponding pixel in the image 54 corresponding to a fourth layer ofthe pyramid image is reflected in the fourth bit of each pixel of thesynthetic image 46-3.

For example, the synthesizing process according to this modified examplewill be described with a focus on the pixels 51A to 54A at the samecoordinates in the images 51 to 54. FIG. 10 is a diagram schematicallyillustrating the process of synthesizing the pixels 51A to 54A.

A bit string with the same number of bits as in the images 51 to 54 isallocated to the pixel 56 in the synthetic image 46-3 at the samecoordinates as the pixels 51A to 54A. In the example illustrated in FIG.10, a bit string of four bits is allocated to the pixel 56 in thesynthetic image 46-3. The value of the pixel 51A in the image 51 of thefirst layer is reflected in the first bit of the pixel 56 in thesynthetic image 46-3. For example, when the pixel 51A has a pixel valueof “0,” “0” is reflected in the first bit of the pixel 56. The value ofthe pixel 52A in the image 52 of the second layer is reflected in thesecond bit of the pixel 56 in the synthetic image 46-3. For example,when the pixel 52A has a pixel value of “0,” “0” is reflected in thesecond bit of the pixel 56. The value of the pixel 53A in the image 53of the third layer is reflected in the third bit of the pixel 56 in thesynthetic image 46-3. For example, when the pixel 53A has a pixel valueof “1,” “1” is reflected in the third bit of the pixel 56. The value ofthe pixel 54A in the image 54 of the fourth layer is reflected in thefourth bit of the pixel 56 in the synthetic image 46-3. For example,when the pixel 54A has a pixel value of “1,” “1” is reflected in thefourth bit of the pixel 56. As a result, the bit string of the pixel 56of the synthetic image 46-3 is (1, 1, 0, 0).

The synthesis part 152 performs this synthesizing process on the pixelsat the same coordinates in the images 51 to 54. As a result, thesynthetic image 46-3 illustrated in FIG. 9 is created. By thissynthesizing process, the synthesis part 152 can synthesize thebinarized images 51 to 54 such that information of the pixel values ofthe pixels is not lost.

The image processing device 100 displays the created synthetic image46-3 in a predetermined display mode. For example, the image processingdevice 100 displays the synthetic image 46-3 while changing a gradationdepending on the pixel value. Alternatively, different colors arecorrelated with the pixel values in advance, and the image processingdevice 100 displays the synthetic image 46-3 while changing the colordepending on the pixel value. By displaying the synthetic image 46-3 inthis way, a user can easily confirm in what layer edge information hasbeen lost and can easily perform program analysis.

The above-description is based on the premise that the synthesis part152 performs binarization and then creates the synthetic image 46, butthe synthesis part 152 may create the synthetic image 46 withoutperforming binarization. In this case, the synthesis part 152 aligns theresolutions of the pyramid image created from the input image 40 andthen synthesizes the pixel values at the same coordinates. Thesynthesizing process employs an arbitrary operation such as a MAXoperation using a maximum pixel value, a MIN operation using a minimumpixel value, or an addition operation of pixel values.

In one or some exemplary embodiments, the image processing device 100stores binarization results of the images 51 to 54 which are the layers.Accordingly, it is possible to easily interpret in which layer an edgeis valid or invalid.

G. Configuration of Image Processing Device 100

A hardware configuration of the image processing device 100 illustratedin FIG. 1 will be described below with reference to FIG. 11. FIG. 11 isa diagram schematically illustrating an example of the hardwareconfiguration of the image processing device 100 according to theembodiment.

The image processing device 100 has a structure based on a generalcomputer architecture and implements various image processing which willbe described later by causing a processor to execute a program which hasbeen previously installed.

More specifically, the image processing device 100 includes a controldevice 110 such as a central processing unit (CPU) or a micro-processingunit (MPU), a random access memory (RAM) 112, a display controller 114,a system controller 116, an input/output (I/O) controller 118, a storagedevice 120, a camera interface 122, an input interface 124, a PLCinterface 126, a communication interface 128, and a memory cardinterface 130. These parts are connected to each other with the centeron the system controller 116 such that data communication with eachother is possible.

The control device 110 implements a target calculation process byexchanging programs (codes) with the system controller 116 and executingthe programs in a predetermined sequence. The system controller 116 isconnected to the control device 110, the Ram 112, the display controller114, and the I/O controller 118 via a bus, exchanges data with theparts, and controls the entire processes of the image processing device100.

The Ram 112 is typically a volatile memory such as a dynamic randomaccess memory (DRAM) and holds a program read from the storage device120, an input image acquired by the imaging part 8, a process result onthe input image, workpiece data, and the like.

The display controller 114 is connected to the display part 102 andoutputs signals for displaying a variety of information to the displaypart 102 in accordance with an internal command from the systemcontroller 116. The display part 102 includes, for example, a liquidcrystal display, or an organic electroluminescence (EL) display.

The I/O controller 118 controls data exchange with a recording medium oran external device which is connected to the image processing device100. More specifically, the I/O controller 118 is connected to thestorage device 120, the camera interface 122, the input interface 124,the PLC interface 126, the communication interface 128, and the memorycard interface 130.

The storage device 120 is typically a nonvolatile magnetic storagedevice and stores various setting values and the like in addition to theimage processing program 20 which is executed by the control device 110.An input image acquired from the imaging part 8 is stored in the storagedevice 120. A semiconductor memory device such as a flash memory or anoptical memory device such as a digital versatile disk random accessmemory (DVD-RAM) may be employed instead of the storage device 120.

The camera interface 122 corresponds to an input part that receivesimage data which is generated by imaging a workpiece W (a searchobject), and relays data transmission between the control device 110 andthe imaging part 8. More specifically, the camera interface 122 can beconnected to one or more imaging parts 8, and an imaging instruction isoutput from the control device 110 to the imaging part 8 via the camerainterface 122. Accordingly, the imaging part 8 shoots a subject andoutputs a created image to the control device 110 via the camerainterface 122.

The input interface 124 relays data transmission between the controldevice 110 and an input device such as a keyboard 104, a mouse, a touchpanel, and a dedicated console. That is, the input interface 124receives an operation command which is issued in response to a user'operation of the input device.

The PLC interface 126 relays data transmission between the controldevice 110 and the PLC 5. More specifically, the PLC interface 126transmits information relevant to a production line state which iscontrolled by the PLC 5, information relevant to a workpiece W, or thelike to the control device 110.

The communication interface 128 relays data transmission between thecontrol device 110 and another personal computer or a server devicewhich is not illustrated. The communication interface 128 typicallyincludes Ethernet (registered trademark), a universal serial bus (USB),or the like. As will be described later, a program downloaded from adelivery server or the like may be installed in the image processingdevice 100 via the communication interface 128 instead of installing aprogram stored in the memory card 106 in the image processing device100. For example, the communication interface 128 receives a signalindicating a state of the imaging part 8 from the imaging part 8, thePLC 5, or the like. The signal indicates whether the imaging part 8 istaking an image.

The memory card interface 130 relays data transmission between thecontrol device 110 and the memory card 106 which is a computer readablerecording medium. That is, the memory card 106 is distributed in a statein which the image processing program 20 which is executed by the imageprocessing device 100 is stored, and the memory card interface 130 readsthe image processing program 20 from the memory card 106. The memorycard interface 130 writes a camera image acquired by the imaging part 8and/or process results in the image processing device 100 to the memorycard 106 in response to an internal command of the control device 110.The memory card 106 includes a general-purpose semiconductor memorydevice such as a secure digital (SD), a magnetic recording medium suchas a flexible disk, or an optical recording medium such as a compactdisk read only memory (CD-ROM).

When a computer having the structure based on a general computerarchitecture is used, an operating system (OS) for providing basicfunctions of the computer may be installed in the computer in additionto applications for providing functions associated with the embodiment.In this case, the image processing program according to this embodimentmay be a program that calls a necessary module among program modulesprovided as a part of the OS in a predetermined sequence and/or time andexecute a processing. That is, the program itself according to theembodiment may not include the above-mentioned module and processing maybe performed in cooperation with the OS. Accordingly, the imageprocessing program according to the embodiment may not include somemodules.

The image processing program according to the embodiment may beincorporated as a part of another program and be provided. In this case,the program itself does not include a module included in the otherprogram which can be combined as described above and processing isperformed in cooperation with the other program. That is, the imageprocessing program according to the embodiment may be incorporated intoanother program.

Alternatively, some or all of functions which are provided by executingthe image processing program may be embodied as a dedicated hardwarecircuit.

H. Control Structure of Image Processing Device 100

A control structure of the image processing device 100 will be describedbelow with reference to FIG. 12. FIG. 12 is a flowchart illustrating apart of a process routine which is performed by the image processingdevice 100. The process routine illustrated in FIG. 12 is embodied bycausing the control device 110 of the image processing device 100 toexecute a program. In another embodiment, some or all of the processroutine may be embodied by a circuit element or other hardware.

In Step S110, the control device 110 acquires a plurality of parametergroups serving as candidates which are set in the image processingprogram 20. The candidates for the parameter groups may be prepared inadvance, may be arbitrarily set by a user, or may be randomly set.

In Step S112, the control device 110 serves as the program executingpart 151 (see FIG. 2) and applies a not-applied parameter group amongthe parameter groups acquired in Step S110 to the image processingprogram 20. Thereafter, the control device 110 executes the imageprocessing program 20 and acquires a pyramid image created in the courseof the execution and search results as execution results. In one or someexemplary embodiments, the control device 110 additionally acquires anexecution time from start of execution of the image processing program20 to end of the execution.

In Step S114, the control device 110 serves as the synthesis part 152(see FIG. 2) and creates a synthetic image from the pyramid imageacquired in Step S112. The method of creating a synthetic image is thesame as described above with reference to FIGS. 5, 7, and 8 and thusdescription thereof will not be repeated.

In Step S120, the control device 110 determines whether the imageprocessing program 20 has been executed for all the parameter groupsacquired in Step S110. When it is determined that the image processingprogram 20 has been executed for all the parameter groups (YES in StepS120), the control device 110 switches the control routine to Step S122.Otherwise (NO in Step S120), the control device 110 returns the controlroutine to Step S112.

In Step S122, the control device 110 displays the synthetic imagesacquired as the result of application of the parameter groups on thedisplay part 102, and displays the search results for the parametergroups on the display part 102 to overlap the corresponding syntheticimages.

In Step S130, the control device 110 determines whether one of thesynthetic images displayed on the display part 102 has been selected.This selecting operation is received by an operation receiving part (forexample, an arbitrary input device such as a mouse, a keyboard, or atouch panel) of the image processing device 100. When it is determinedthat one of the synthetic images displayed on the display part 102 hasbeen selected (YES in Step S130), the control device 110 switches thecontrol routine to Step S132. Otherwise (NO in Step S130), the controldevice 110 performs the process of Step S130 again.

In Step S132, the control device 110 serves as the setting part 153 (seeFIG. 2) and applies the parameter group corresponding to the selectedsynthetic image as setting parameters.

I. Conclusion

As described above, the image processing device 100 applies a pluralityof predetermined parameter groups to the image processing program 20 forexecuting a pyramid algorithm. The image processing device 100 acquiresa search result of the image processing program 20 for each parametergroup and acquires a pyramid image which is obtained in the course ofexecution of the image processing program for each parameter group.

Thereafter, the image processing device 100 creates a synthetic imagefrom the pyramid image acquired for each parameter group, displays thesynthetic images, and displays the search result for each parametergroup to overlap the corresponding synthetic image. The image processingdevice 100 is configured to select one of the displayed syntheticimages. The image processing device 100 sets the parameter groupcorresponding to the selected synthetic image as the setting parameter22 of the image processing program 20.

A user can intuitively understand whether each parameter group is betterby visually confirming the synthetic images and the search results.Then, the user can select the parameter group with which good searchaccuracy can be obtained. Accordingly, a user not having any knowledgeof the image processing program 20 can set an optical parameter groupand can support setting of parameters for the image processing program20. These can act particularly effectively when the number of types ofparameters to be set is large or when the total number of pyramid imagesis large.

The above-disclosed embodiments should be understood to be merelyexemplary, but not restrictive in all aspects. The scope of theinvention is defined by the appended claims, not by the abovedescription, and is intended to include all modifications withinmeanings and scopes equivalent to the scope of the claims.

According to an embodiment, the image processing program searches thesearch object sequentially from an image with a lowest resolution in theimage group with different resolutions which is created from the inputimage and searches the search object while restricting a search range inan image with a next higher resolution on the basis of the search resultin each image.

According to an embodiment, the parameter group which is applied to theimage processing program includes a threshold value group that is usedto determine whether the search object is included in each image as thethreshold value group which is applied to each image of the image groupwith different resolutions which is created from the input image.

According to an embodiment, the display part additionally displays anevaluated value for each of the one or more parameter groups.

According to an embodiment, the synthesis part adjusts the resolutionsof the image group with different resolutions which is created from theinput image and then creates a synthetic image from the image group.

According to an embodiment, the image processing program binarizes theimage group with different resolutions which is created from the inputimage into pixels indicating an edge and pixels not indicating an edge.The synthesis part aligns the resolutions of the binarized image group,performs an AND operation on pixel values at the same coordinates in theimage group, and outputs the synthesis image as the result of the ANDoperation.

According to an embodiment, the image processing program binarizes theimage group with different resolutions which is created from the inputimage into pixels indicating an edge and pixels not indicating an edge.The synthesis part aligns the resolutions of the binarized image group,performs an OR operation on pixel values at the same coordinates in theimage group, and outputs the synthesis image as the result of the ORoperation.

According to an embodiment, the image processing program binarizes theimage group with different resolutions which is created from the inputimage into pixels indicating an edge and pixels not indicating an edge.The synthesis part aligns the resolutions of the binarized image group,synthesizes pixels at the same coordinates in the image group such thatinformation of pixel values is not lost, and outputs the synthesis imageas the result of the synthesis.

According to an embodiment, the parameter group which is applied to theimage processing program includes a threshold value group that is usedto determine whether each pixel in each image is a pixel indicating anedge as a threshold value group which is applied to each image of theimage group with different resolutions which is created from the inputimage.

What is claimed is:
 1. An image processing device comprising: an image acquiring part that acquires an input image including a search object; a program executing part that applies one or more predetermined parameter groups to an image processing program which searches the search object using an image group with different resolutions which is created from the input image, acquires a search result from the image processing program for each of the one or more parameter groups, and acquires the image group which is acquired in the course of execution of the image processing program for each of the one or more parameter groups; a synthesis part that creates a synthetic image from each of the acquired one or more image groups; a display part that displays the created one or more synthetic images and displays the acquired one or more search results to overlap the one or more synthetic images; an operation receiving part that receives an operation of selecting one of the one or more synthetic images displayed on the display part; and a setting part that sets the parameter group corresponding to the synthetic image selected by the operation in the image processing program.
 2. The image processing device according to claim 1, wherein the synthesis part adjusts the resolutions of the image group with different resolutions which is created from the input image and then creates a synthetic image from the image group.
 3. The image processing device according to claim 1, wherein the display part additionally displays an evaluated value for each of the one or more parameter groups.
 4. The image processing device according to claim 3, wherein the synthesis part adjusts the resolutions of the image group with different resolutions which is created from the input image and then creates a synthetic image from the image group.
 5. The image processing device according to claim 1, wherein the image processing program searches the search object sequentially from an image with a lowest resolution in the image group with different resolutions which is created from the input image and searches the search object while restricting a search range in an image with a next higher resolution on the basis of the search result in each image.
 6. The image processing device according to claim 5, wherein the synthesis part adjusts the resolutions of the image group with different resolutions which is created from the input image and then creates a synthetic image from the image group.
 7. The image processing device according to claim 5, wherein the display part additionally displays an evaluated value for each of the one or more parameter groups.
 8. The image processing device according to claim 7, wherein the synthesis part adjusts the resolutions of the image group with different resolutions which is created from the input image and then creates a synthetic image from the image group.
 9. The image processing device according to claim 5, wherein the parameter group which is applied to the image processing program includes a threshold value group that is used to determine whether the search object is included in each image as the threshold value group which is applied to each image of the image group with different resolutions which is created from the input image.
 10. The image processing device according to claim 9, wherein the synthesis part adjusts the resolutions of the image group with different resolutions which is created from the input image and then creates a synthetic image from the image group.
 11. The image processing device according to claim 9, wherein the display part additionally displays an evaluated value for each of the one or more parameter groups.
 12. The image processing device according to claim 11, wherein the synthesis part adjusts the resolutions of the image group with different resolutions which is created from the input image and then creates a synthetic image from the image group.
 13. The image processing device according to claim 12, wherein the image processing program binarizes the image group with different resolutions which is created from the input image into pixels indicating an edge and pixels not indicating an edge, and wherein the synthesis part aligns the resolutions of the binarized image group, performs an AND operation on pixel values at the same coordinates in the image group, and outputs the synthesis image as a result of the AND operation.
 14. The image processing device according to claim 13, wherein the parameter group which is applied to the image processing program includes a threshold value group that is used to determine whether each pixel in each image is pixel indicating an edge as a threshold value group which is applied to each image of the image group with different resolutions which is created from the input image.
 15. The image processing device according to claim 12, wherein the image processing program binarizes the image group with different resolutions which is created from the input image into pixels indicating an edge and pixels not indicating an edge, and wherein the synthesis part aligns the resolutions of the binarized image group, performs an OR operation on pixel values at the same coordinates in the image group, and outputs the synthesis image as a result of the OR operation.
 16. The image processing device according to claim 15, wherein the parameter group which is applied to the image processing program includes a threshold value group that is used to determine whether each pixel in each image is pixel indicating an edge as a threshold value group which is applied to each image of the image group with different resolutions which is created from the input image.
 17. The image processing device according to claim 12, wherein the image processing program binarizes the image group with different resolutions which is created from the input image into pixels indicating an edge and pixels not indicating an edge, and wherein the synthesis part aligns the resolutions of the binarized image group, synthesizes pixels at the same coordinates in the image group such that information of pixel values is not lost, and outputs the synthesis image as the result of the synthesis.
 18. The image processing device according to claim 17, wherein the parameter group which is applied to the image processing program includes a threshold value group that is used to determine whether each pixel in each image is pixel indicating an edge as a threshold value group which is applied to each image of the image group with different resolutions which is created from the input image.
 19. A setting support method of supporting setting of parameters for an image processing program, the setting support method comprising: a step of acquiring an input image including a search object; a step of applying one or more predetermined parameter groups to an image processing program which searches the search object using an image group with different resolutions which is created from the input image, acquiring a search result from the image processing program for each of the one or more parameter groups, and acquiring the image group which is acquired in the course of execution of the image processing program for each of the one or more parameter groups; a step of creating a synthetic image from each of the acquired one or more image groups; a step of displaying the created one or more synthetic images and displaying the acquired one or more search results to overlap the one or more synthetic images on a display part; a step of receiving an operation of selecting one of the one or more synthetic images displayed on the display part; and a step of setting the parameter group corresponding to the synthetic image selected by the operation in the image processing program.
 20. A non-transitory computer-readable media, compressing a setting support program for supporting setting of parameters for an image processing program, the setting support program causing an image processing device to perform: a step of acquiring an input image including a search object; a step of applying one or more predetermined parameter groups to an image processing program which searches the search object using an image group with different resolutions which is created from the input image, acquiring a search result from the image processing program for each of the one or more parameter groups, and acquiring the image group which is acquired in the course of execution of the image processing program for each of the one or more parameter groups; a step of creating a synthetic image from each of the acquired one or more image groups; a step of displaying the created one or more synthetic images and displaying the acquired one or more search results to overlap the one or more synthetic images on a display part; a step of receiving an operation of selecting one of the one or more synthetic images displayed on the display part; and a step of setting the parameter group corresponding to the synthetic image selected by the operation in the image processing program. 